Skip to content

fix(agc): stop optional compute captures at their bounds - #6259

Merged
adriadam10 merged 2 commits into
boykopovar:mainfrom
fdfawef324:codex/optional-compute-capture-stop
Oct 18, 2026
Merged

adriadam10 merged 2 commits into
boykopovar:mainfrom
fdfawef324:codex/optional-compute-capture-stop

Conversation

@fdfawef324

@fdfawef324 fdfawef324 commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

What

When APS5_DUMP_COMPUTE_PIPELINES reaches its 128-record or 256 MiB bound, the strict writer throws inside compute pipeline creation. The optional dispatch capture now stops once with a diagnostic at either bound or a host I/O failure, and skips subsequent capture filtering and SPIR-V copies. The strict Write API retains its bounds, deduplication, immutable records and diagnostics. Invalid pipeline input, damaged/incomplete records, unexpected entries and allocation failures still propagate.

PPSA01342 reached the generic error compute pipeline capture: capture directory reached 128 entries at 1385.2 seconds in the historical external experimental integration run ds-c0076. Its metadata, triage and fatal line were checked. This title observation is scoped to that experimental integration; current-main title execution has not been verified for this change.

Tested

Windows Server 2025 10.0.26100, O3 CPU tests using the actual production capture writer and repository homebrew fixture:

  • Existing codec, immutable writes, malformed inputs and 5,500,000-word record tests pass.
  • Actual 128-record boundary, deduplication at the cap, and stop on the next distinct record pass (1.30 s). Exactly 256 MiB is accepted and the next distinct record stops capture (1.19 s), with production limits unchanged. The original main writer fails both boundary scenarios.
  • A real owned output-path conflict stops optional capture once; strict writes retain their diagnostic. Malformed input and damaged, missing, nonregular or foreign records remain errors; later optional calls do not touch their output directory.
  • Selective broad-catch, omitted disabled validation and rethrown optional-I/O controls fail their intended assertions. Private injected open/write/close/read failures and allocation failure verify the production wrapper's error policy.
  • The literal dispatch gate bypasses 1024 subsequent copy/filter attempts after a real I/O stop; removing its enabled check fails. Whole VulkanDevice.cpp passes O3 syntax checking with the matching component headers.

Current head 66556448b (rebased onto main 38d245448, patch unchanged), MinGW GCC 15.2 Release, NVIDIA GeForce RTX 3060 12 GB, driver 591.86:

  • libSceAgcDriver builds. All seven agc_compute_pipeline* CTests pass with ANYPS5_REQUIRE_VULKAN=1 and ANYPS5_ENABLE_SPIRV_TOOLS=ON, including agc_compute_pipeline_capture_run: 2 pipelines are captured through the dispatch path and replayed without dispatch.
  • The repository's branch-past-endpgm compute fixture is run with APS5_DUMP_COMPUTE_PIPELINES set to a directory already holding 128 entries, a directory at exactly 256 MiB, or a path below a regular file. On this head each run passes, prints one optional capture disabled line naming the bound or I/O error, and adds no records. With main's writer and dispatch gate the same three runs fail: the 128-entry and 256 MiB errors end the fixture with exit 1, and the I/O case raises an undeliverable filesystem_error (exit 67). An empty directory receives the same 2 records on both.

No title run or title-performance claim.

Checklist

  • Based on current main; no other open PR implements the same functions
  • One topic per PR; follow-ups go in a new PR
  • No comments in code except technical debt
  • Unimplemented paths throw (NotImplemented_nid_no_patch); silent stubs are listed under Silent stubs in their library's technical debt file
  • No notes, investigation .md files or images added to the repository (attach them to this PR)
  • New third-party code is a submodule built from source
  • Generic behaviour, not specific to one title
  • Depends on: none; fix(agc): disable optional BMP dumps after host I/O failures #5800 addresses the separate optional BMP worker
  • Unblocks: PPSA01342 Demon's Souls (PPSA01341) #1283 (optional-capture stop observed on the external experimental integration above)
  • GPUs tested: NVIDIA GeForce RTX 3060 12 GB, driver 591.86 (32.0.15.9186), Vulkan 1.4.325, Windows Server 2025 10.0.26100; compute capture fixture and replay runs above, no title run
  • AI-assisted: yes

Co-Authored-By: Codex noreply@openai.com

@jotitadev jotitadev left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

macOS run of the capture tests, plus one macOS-specific case the optional path still turns into a fatal error.

Machine: Apple M2 Max, macOS 27.2, Apple clang (Xcode 27), libc++. Head 870952f2c, base main 8856f180c. main does not build fully on macOS, so I compiled agc_compute_pipeline_capture_tests the way its CMake target does (tests/ComputePipelineCapture.cpp + Execution/src/ComputePipelineCapture.cpp, -std=c++20 -O2, Vulkan headers from the SDK) as a standalone binary, natively on arm64 and as x86_64 under Rosetta.

  • All five modes pass on both architectures: default (0.29 s arm64 / 0.60 s x86_64), --optional-entry-cap, --optional-byte-cap (0.70 s / 0.77 s), --optional-io (cannot create capture directory: Not a directory), --optional-invalid.
  • The PR's test does not compile against the base writer (WriteOptional/OptionalEnabled are new), as expected.

Finder writes a .DS_Store file into folders it opens. The directory scan only accepts subdirectories, so that file hits capture directory contains an unexpected entry, which is a CaptureError, not a limit or I/O failure. WriteOptional rethrows it, and the next distinct compute pipeline creation fails. Reproduced on this head: one WriteOptional, then an empty .DS_Store in the capture directory, then WriteOptional with a different specialization value throws compute pipeline capture: capture directory contains an unexpected entry. The check is the same on main, so this is not a regression, but on macOS just looking at APS5_DUMP_COMPUTE_PIPELINES in Finder during a run is enough to make the next pipeline creation fail. Skipping entries whose name starts with . in the scan, or treating an unexpected entry as an optional stop, would avoid it. Your call whether that fits this PR or a follow-up.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@fdfawef324
fdfawef324 force-pushed the codex/optional-compute-capture-stop branch from 870952f to 6655644 Compare October 15, 2026 12:05
@fdfawef324

Copy link
Copy Markdown
Contributor Author

@jotitadev thanks for the macOS run. I rebased onto main 38d245448, and the range-diff shows both commits unchanged, so your results for 870952f2c also cover the new head 66556448b.

On .DS_Store: I can confirm this. The scan rejects every entry that is not a directory, whatever its name. This PR keeps that an error on purpose. The --optional-invalid test checks that a stray file named foreign still throws and leaves optional capture enabled. Only the two bounds and host I/O failures stop optional capture. Skipping dot-prefixed entries would change which directories both Write and WriteOptional accept as capture directories. That behaviour already exists on main and is separate from the bounds, so it belongs in a follow-up PR once this one lands, not in this one.

I've also added current-head RTX 3060 results to the description. All seven agc_compute_pipeline* CTests pass, including capture_run. The real compute fixture also passes when the capture directory is already at 128 entries, already at 256 MiB, or under a regular file, and the same three runs fail when built from main's writer.

AI-assisted: yes

@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Some comparisons could not be completed because a pull request conflicts with the base branch. Review these shared files manually before concluding that there is no overlap.

Pull request Shared files not compared
PR 1341 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1368 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 1395 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 2434 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4040 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4072 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4472 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4650 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 4891 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5021 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5371 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5426 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 5441 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5499 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 5925 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6044 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6177 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6185 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6186 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6188 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6192 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6234 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6292 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6300 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6360 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6414 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6454 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6486 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6501 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6503 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6535 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6615 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6701 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6747 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6763 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6788 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6808 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6856 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6868 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6937 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6941 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6946 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6947 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 6963 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 6977 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7201 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7223 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7332 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7390 core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp
PR 7395 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7615 core/libs/prx/libSceAgcDriver/CMakeLists.txt
PR 7642 core/libs/prx/libSceAgcDriver/CMakeLists.txt
core/libs/prx/libSceAgcDriver/Execution/src/VulkanDevice.cpp

@jotitadev

Copy link
Copy Markdown
Contributor

Thanks, that makes sense. Re-ran 66556448b on the M2 Max: all 5 capture modes pass on arm64 and on x86_64 under Rosetta, and the .DS_Store case still throws capture directory contains an unexpected entry, as you describe.

I'll take the .DS_Store follow-up once this lands. It will skip only Finder's .DS_Store (not every dot entry) in the scan that Write and WriteOptional share, keep --optional-invalid throwing for other stray files, and add a test for it.

Written with AI assistance (Claude Code). I ran the builds and tests above on my machine.

@coltken95

Copy link
Copy Markdown

Tested on Linux x86-64 (Release build, lavapipe / llvmpipe LLVM 20.1.2), with 6655644 merged into main 4eb91ea:

  • With the PR: agc_compute_pipeline_capture and the four new agc_compute_pipeline_capture_optional_* tests (byte_cap, entry_cap, invalid, io) pass.
  • No comparison against main: ComputePipelineCapture.cpp doesn't build without the change ('struct AgcDriver::ComputePipelineCapture' has no member named 'WriteOptional').

AI-assisted: yes (Claude Code)

@github-actions github-actions Bot added the needs rebase Conflicts with main, rebase needed label Oct 16, 2026
@fdfawef324
fdfawef324 force-pushed the codex/optional-compute-capture-stop branch from 6655644 to 0984c30 Compare October 16, 2026 19:52
@fdfawef324

Copy link
Copy Markdown
Contributor Author

We rebased this onto current main (c842491); the new head is 0984c30 (was 6655644), the same two commits with their authorship unchanged.
The only conflict was in VulkanDevice.cpp: main moved the compute pipeline build into createComputePipeline, so the OptionalEnabled gate and the WriteOptional call moved there with no other change.
Tests: the five agc_compute_pipeline_capture tests (strict round trip, plus entry-cap, byte-cap, io and invalid) pass on this head on CPU, and tools/check_conventions.py --base upstream/main passes.
GPU, on the RTX 3060 (driver 591.86, Windows Server 2025): agc_compute_pipeline_capture_run and agc_compute_pipeline_replay pass (2 of 2) on this head.
AI-assisted: yes

@adriadam10 adriadam10 self-assigned this Oct 18, 2026
fdfawef324 and others added 2 commits October 18, 2026 21:29
Preserve strict capture validation and immutable writes while disabling
optional dispatch captures after capacity or host I/O failures.

Co-Authored-By: Codex <noreply@openai.com>
Keep malformed record structures outside the optional host I/O boundary.

Co-Authored-By: Codex <noreply@openai.com>
@adriadam10
adriadam10 force-pushed the codex/optional-compute-capture-stop branch from 0984c30 to a437891 Compare October 18, 2026 19:50
@github-actions github-actions Bot added unblocks title Names a title ID on the Unblocks line and removed needs rebase Conflicts with main, rebase needed labels Oct 18, 2026
@adriadam10
adriadam10 merged commit 4d1d947 into boykopovar:main Oct 18, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agc libSceAgc* build unblocks title Names a title ID on the Unblocks line

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants